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Abstract. Using analytical methods of statistical mechanics, we analyse the typical 
behaviour of a multiple-input multiple-output (MIMO) Gaussian channel with binary 
inputs under LDPC network coding and joint decoding. The saddle point equations 
for the replica symmetric solution are found in particular realizations of this channel, 
including a small and large number of transmitters and receivers. In particular, we 
examine the cases of a single transmitter, a single receiver and the symmetric and 
asymmetric interference channels. Both dynamical and thermodynamical transitions 
from the ferromagnetic solution of perfect decoding to a non-ferromagnetic solution are 
identified for the cases considered, marking the practical and theoretical limits of the 
system under the current coding scheme. Numerical results are provided, showing 
the typical level of improvement/deterioration achieved with respect to the single 
transmitter/receiver result, for the various cases. 
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1. Introduction 

The statistical physics of disordered systems has been systematically developed over 
the past few decades to analyse systems of interacting components under different 
interaction regimes [HE]- It enables one to derive typical macroscopic properties of 
systems comprising a large number of units under conditions of quenched disorder, 
which correspond to different randomly sampled instances of the problem. 

While their origin lies in the study of spin glasses P SI |5], methods of statistical 
mechanics have been successfully employed to study a broad range of interdisciplinary 
subjects, from thermodynamics of fluids to biological and even sociological problems. 
In these studies, the problems were mapped onto known statistical physics models, such 
as Ising spin systems, and analysed using established methods and techniques from 
statistical physics. 

In particular, these methods have been successfully employed recently to investigate 
hard computational problems P, [7j as well as problems in information theory [8] and 
multi-user communication [9]. They proved to be highly useful for gaining insight 
into the properties of the problems studied and in providing exact typical case results 
that complement the rigorous bounds reported in the theoretical computer science and 
information theory literature. 

In the current study we employ the powerful analytical methods of statistical 
mechanics to examine the typical properties of Multiple- Input Multiple-Output (MIMO) 
communication channels where messages are encoded using state of the art Low-Density 
Parity-Check (LDPC) error correcting codes ^TOi \TT\ [T2l [13] . 

MIMO channels are becoming increasingly more relevant in modern communication 
networks that rely on adaptive and ad-hoc configurations. Sensor networks, for 
instance, may rely on simultaneous transmission of information from a large number 
of transmitters that give rise to high levels of interference; while multiple access, at 
various levels, is exercised daily by millions of mobile phone users. 

This problem of communication over a MIMO channel is particularly amenable to 
a statistical physics based analysis for the following reasons: Firstly, previous studies 
in the areas of LDPC error-correcting codes [8] and Code Division Multiple Access 
(CDMA) P, [H] paved the way for the study of MIMO systems; and secondly, the 
framework of multi-user communication channels is difficult to analyse using traditional 
methods of information theory [15], but can be readily accommodated within the 
statistical physics framework, particularly in the case of a large number of users. 

The paper is organised as follows: In section [2] we introduce the model to be 
analysed, followed by statistical physics framework in section [3l We then study several 
communication channels: a single transmitter and multiple receivers in section HI 
multiple access in [5] and symmetric and asymmetric interference channels in section |6l 
In each of the sections we will consider both cases of a small and large number of users. 
We conclude with general insights and future directions. 
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As the communication model considered is based on LDPC codes we will briefly 
introduce their main characteristics, within the single channel setting, before describing 
the MIMO communication channel to be studied. 

LDPC codes, introduced originally by Gallager [10], are used to encode A^- 
dimensional message vectors s into M-dimensional codewords t. They are deflned 
by a binary matrix A = [Ci \ C2], called parity-check matrix, concatenating two 
very sparse matrices known to both sender and receiver, with C2 (of dimensionality 
{M - N) X {M - N)) being invertible and Ci of dimensionality {M - N) x N. The 
matrix A can be either random or structured, characterised by the number of non-zero 
elements per row/column. Irregular codes show superior performance with respect to 
regular constructions [HI [16] if they are constructed carefully. However, to simplify the 
presentation, we focus here on regular constructions; the generalisation of the methods 
presented here to irregular constructions is straightforward [TTj ITS]. 

Encoding refers to the mapping of a A^- dimensional binary vector s G {0,1}^ 
(original message) to M-dimensional codewords t G {0, 1}*^ (M > A^) by the linear 
product 

t = Gs (mod 2) , (1) 

where all operations are performed in the fleld {0, 1} and are indicated by (mod 2). The 
generator matrix is of the form 

(mod 2) , (2) 

where / is the N x N identity matrix. By construction AG = (mod 2) and the flrst 
A^ bits of t correspond to the original message s. 

Decoding is carried out by estimating the most probable transmitted vector from 
the received corrupted codeword [ITJ [H] . 

In this work, we analyse a MIMO Gaussian channel with L sender and O receiver 
units. In this channel, L original binary messages Si G {0, 1}^, i = 1, L are encoded 
using LDPC error- correcting codes with independently chosen parity-check matrices Ai 
for each message into binary codewords ti G {0, 1}*^. 

Note that, both messages Sj and codewords ti are vectors and should include two 
different indices, the bit index and a separate index for the number of senders/receivers 
{i). For brevity, we will reserve the boldface notation for denoting the sets in the 
sender /receiver indices and will explicitly denote the bit index. 

We concentrate here on regular Gallager codes, with exactly K non-zero elements 
per row and C non-zero elements per column in the parity check matrix, which obey 
the relation C = (1 — R)K, where R = N/M is the code rate. The codewords are 
transmitted in discrete units of time. 



G 
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In order to apply the tools of statistical mechanics, we use, for mathematical 
convenience, the transformation 

X - (-l)^ (3) 

to map the Boolean variables tj G {0, 1}*''^ onto spin variables ti G {1, —1}^^. Although 
they are different variables, we denote both with the same letter tj. The appropriate 
use of each one of them will be clear from the context. At each discrete time step /i, 
the (already mapped) vector f^, fi = I, ...,M is transmitted and corrupted by additive 
white Gaussian noise (AWGN) obeying the equation 

r'^ = Stf" + i/^, (4) 

where S" is an O x L matrix with elements Sji, and the Gaussian noise, independent of 
the time, is given by the vector i^^ = (i^f, Uq) with ~ J\f{0, cr|), j = 1, O, V/x, 
i.e., of zero mean and variance (t|. 

The matrix 5", which we call the interference matrix, plays an essential role in the 
current analysis as it crosses messages between senders and receivers and is responsible 
for important interference effects. 



3. Replica Analysis 

The statistical mechanics based analysis focuses on the decoding process as it is directly 
linked to the Hamiltonian within the physics framework |T9] . 

Decoding is carried out along the same lines as in LDPC error- correcting codes; the 
estimate of the first N bits of the codeword, which contain the original uncoded message, 
will be made by introducing L dynamical variable values G {±1}^, representing 
candidate vectors for each of the transmitted codewords. These will eventually give rise 
to the estimate of the various codewords {ti} , i = 1, . . . L, hj the O receivers, each of 
which has access to all the received messages. 

In the statistical analysis, we are interested in the behaviour averaged over 
the system's disorder, given by the quenched variables r, all possible encodings (or 
equivalently, all parity-check matrices Ai, for each sender) and all transmitted codewords 
ti. 

If we allow some degree of error in the decoding, in the form of a prior error 
probability, the estimator which minimises the bit error probability is the Marginal 
Posterior Maximiser (MPM) for each dynamical variable pOf IT8]. 

t7 = sgn(rf)p(T.|^), (5) 

where r = (n, ...,tl). 

The expected overlap between the estimated and the transmitted codewords serves 
as a quality measure for the error correction performance 

1 
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where the average is taken over the joint probabihty distribution V{Ai, AL,r,t). 
These performance measures will also be indicative of the dynamical transition from the 
ferromagnetic solution of perfect decoding to a non-ferromagnetic solution as reflected 
in their values. Note that the receiver will only get all the messages correctly if = 1 
for all i at the same time. 

The free-energy in the thermodynamic limit M 

1 



oo is given by 



where Z is the partition function 

o 

T L i=i 
with the Hamiltonian component for each receiver j 

2 



(7) 



T r 



n,{T\v) 



(8) 



The Hamiltonian gives rise to a likelihood term for the agreement between the received 
aggregated vector and the candidate codewords. The decoding temperature [3 is 
considered the same for every receiver and each obeys the parity-check constraint, 
which for the spin variables is defined by 

M 



1 M-N. 



The decoding process is aimed at maximising the probability 

o 

-(3Y,n,{T\v) 



1 



V{t\y) = — exp 



(9) 



(10) 



To calculate (InZ)^^ A^rt thermodynamic limit, where M,N — > oo while 

keeping the code rate R = N/M constant, we use the replica method [H [2] which relies 
on the identity 

ain(Z") 



{\nZ) 



lim 

n-*0 



dn 



11) 



and employs an analytical continuation of integer values of n to a real value that 
approaches zero. The calculations will follow the same guidelines as in [IS] and we 
refer the reader to the appendix for further details. 
The partition function is given by 

2" 



z 



E 



" L 






exp 







O M 






(12) 



where x is an indicator function, which is zero if does not obey the parity-check 
equations defined by the matrix Ai. 
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We assume that the matrices Ai are chosen from the same ensemble of parity-check 
matrices, which means that all code rates will be the same, Ri = R. From information 
theoretical considerations, the capacity region is then given by 

aR < C, (13) 

where a = L/0 is a characteristic constant of the system called its load and C, the 
capacity with joint decoding for an arbitrary distribution of inputs, is obtained by 
conventional information theoretical methods |15jJ and is given by 

C = ^log,detiIo + SS^C;'), (14) 

where T indicates transposition, Iq is the 0-dimensional unit matrix and is an O- 
dimensional square diagonal noise matrix given by {Ci,)jk = cr'j6jk- This result will be 
used as a benchmark and an upper bound for our results 



4. Single Transmitter 



In this and the following sections we compare the replica symmetric results with the 
known information theoretical limits. The case L = O = 1 is easily seen to recover 
the usual results for a simple Gaussian channel as obtained in [18]|. In the particular 
case of one sender and an arbitrary number of receivers, the channel matrix is an O- 
dimensional column vector. The Replica Symmetric (RS) saddle point equations are 
(see Appendix A.l ) 

K-l 



TC(XI 



6 \ X — Y\ ^ 

1=1 



6\ X — tanh 



atanh x^ + (3 ^ 



1=1 



(15) 
(16) 



with 



o 



r-\{U{S,,a^), (17) 

i=i 

and where the averages ()^ and ()^ are taken with respect to the distributions 7r(x) and 
7r(x), respectively. 

The overlap is given by 

d =(sgn(p)) with 



p(p) = / (5 p - tanh 



■ c o ^.^ 1\ 

^ atanh £' + /3 ^ 

.1=1 i=i J/ 



(18) 
(19) 



The free-energy is 
C 



c 



K 



K 



ln2 + C(ln(l + xx))^^^ _ _^ In [ 1 + J] 



m=l 
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- ( In < ^exp 



=±1 



c 




Uil + TX^)}) . (20) 



x,r 



C = -log2(l + - ). (23) 



The ferromagnetic solution, 

7r(a;) = 6{x — 1), and n{x) = 6{x — 1), (21) 

represents perfect decoding; it is always present for all noise levels and has free-energy 
/ = 0/2. 

The internal energy and the entropy can be derived from the free energy by the 
well-known relations 

u = -^{Pf), s = (3{u-f). (22) 

Let us study the symmetric case where all transmitters emit with the same unit 
power, all entries of S are equal to 1, and all receivers experience the same noise level 
(T^. The capacity, as given by equation (IT^ . is 

and the Shannon limit of perfect decoding is attained when R = C, giving for the 
threshold noise the result 

To obtain numerical solutions for the various cases we iterated the saddle-point 
equations (ITSl) . using population dynamics, and then calculated the quantities of interest 
such as the overlap d, the free energy / and the entropy s of equations (fT8r - (l22l) . 

Figure [1] shows the overlap for L = 1 (one sender) ,0 = 2 (two receivers) , cr| = 
(equal noise level for all receivers), Sj = 1 and R = 1/4 (with K = 4 and C = 3) at the 
Nishimori temperature (3=1. The choice of the Nishimori temperature simplifies the 
analysis as it is known that for this temperature, the system does not enter the spin- 
glass phase [1]. Similar to the case of LDPC codes, there is no difference between the 
RS results, obtained using the Nishimori condition, and those obtained using the replica 
symmetry breaking ansatz for the noisy channel studied here [211 [22] : this motivates 
our present choice of the replica symmetric ansatz. 

We can see that the overlap has the value 1 up to the noise level termed the 
dynamical transition point. This means that while the noise level is kept below 
this point, all the receivers can perfectly recover the transmitted message as the 
ferromagnetic solution is the only stable solution. The ferromagnetic solution remains 
dominant between this point and the thermodynamical transition point., which marks 
the noise level where the non-ferromagnetic state becomes dominant; although an 
exponential number of sub-optimal stable solutions in this range prevent the iterative 
population dynamics from converging to the ferromagnetic solution (starting from an 
arbitrary initial state). 

The entropy plot in the inset clarifies the type of solutions obtained as the noise 
level increases: the entropy is zero up to the dynamical transition, meaning that the only 
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Figure 1. Overlap in the single-sender case for O — 2. The solid line describes 
the result obtained by iterating the saddle point equations (llSp from arbitrary 
initial conditions. The dotted-dashed line shows the theoretical limit obtained from 
equation ((24|) and the dashed line shows the theoretical limit for sending a doubled 
message via a single Gaussian channel. The inset shows a plot of the entropy; the point 
where the entropy becomes negative marks the emergence of metastable states and the 
dynamical transition point, while the point where it crosses back the zero entropy line 
marks the thermodynamical transition noise value. 



stable state is the ferromagnetic one. Metastable suboptimal solutions emerge above 
this point which could be explored using the replica symmetry breaking ansatz [2n 122] ; 
these contribute to (unphysical) negative entropy values in this range |18]. The point 
where the entropy line crosses the coordinate axis coincides with the thermodynamical 
transition point. The thermodynamical transition is always upper bounded by the 
Shannon theoretical limit, which is also shown in the overlap plot as a vertical dashed 
line. 

In tabled] we compare the theoretical limit of sending the same message O times via 
a simple Gaussian channel (one sender and one receiver) with noise level equal to the one 
considered here (second column) with the theoretical limit for the MIMO channel given 
by equation (|24ll (third column) and the points of the dynamical (fourth column) and the 
thermodynamical (fifth column) transitions obtained by numerical integration of the RS 
equations for O = 1, 2, 3 receivers. It is clear that the dynamical and thermodynamical 
transitions occur always before the theoretical limit. As expected, the more receivers 
are added, the higher the noise level the system can tolerate. However, the differences 
between the dynamical and the thermodynamical transition values, and between the 
thermodynamical transition and theoretical limit increase. Both are related to the fact 
that, in adding more receivers, we also increase the number of metastable states in the 
system; these emerge earlier and contribute to a higher entropy. 
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Comparing the theoretical noise hmit for sending the message O times by a simple 
Gaussian channel with the limit for the MIMO channel with one sender and O receivers, 
we can see that the later is just O times the former. This can be understood noting 
that the information being sent in the MIMO channel is the same as in the 0-replicated 
Gaussian channel, but with O times the power; while in the MIMO channel case, the 
O bits are sent with power 1 at each time step. We can see by that the results of the 
RS ansatz that the transition points are even below the theoretical limit for the simple 
Gaussian channel and significantly below the MIMO limit. This clearly shows that in 
this type of communication channel, even with joint decoding, the available information 
is being poorly used. It makes a strong case for the use of network coding, i.e., to encode 
jointly the vectors prior to transmission. 

Network coding, for instance using fountain codes [23l[24j . is likely to make a better 
use of the resource by generating codewords that are more suited for better extraction 
of information under joint decoding. 

For O > 3 the numerical instabilities grow larger with O and a precise evaluation 
of the points is increasingly more difficult. 



Table 1. Comparison between the Shannon hmit for a simple Gaussian channel and 
the MIMO channel, the dynamical transition point and the thermodynamical transition 
for the single-sender case (L = 1). 






Shannon's Limit 
(Gaussian Channel) 


Shannon's Limit 
(MIMO Channel) 


Dynamical 
Transition 


Thermodynamical 
Transition 


1 


2.41 


2.41 


1.59 


2.24 


2 


5.28 


10.57 


3.28 


4.59 


3 


8.17 


24.50 


4.90 


6.68 



Another case of interest is that of an infinite number of receivers. When O ^ oo, 
the average over the r variables in equation (fTSll . can be substituted by an average over 
the Gaussian variable 

i=i J 

In the case, of equal noise and Sj = 1, this variable has zero mean and variance O/cr^ 
which refiects the signal to noise ratio appearing in the capacity expression 



5. Multiple Access Channel 

The multiple access channel (MAC) is a particular case where = 1 and S is an L- 
dimensional row matrix. Let us consider once more the symmetric case where Sji = 1 
and cr| = (7^. The capacity then becomes 

C = 1 log2 (l + ^) , (26) 
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and the threshold noise 

= ^si^n- (27) 

In this case, due to the interference effect in the received message, one should 
guarantee that the interference term has the correct order with respect to L. Taking 
into account that the received messages are independent random variables, we normalise 
their sum by the factor of 1 / 

The simplest case is L = 2 and the RS saddle point equations for user 1 are given 

by 

K-l 



vri(xi) 



+-ln 
2 



X 1 



(2^ 



1=1 



X 

'C-1 



6 X 



tanh I y^atanhx'i H ^-—;= 



1 — tanh (2^2) tanh ( ^i^i atanhi;2 



)3r 



1 + tanh {-^^ tanh ^^[i^ atanhxg + 




and the overlap is 

di = (sgn (p))^, 

V{p) = {6 (p 



(29) 



(30) 



c 



tanh I atanhx'i H — ^—f= 



+-ln 
2 



1 — tanh (2^5-) tanh iYlf=i atanhx2 + 



1 + tanh {^) tanh {Y^fLi atanhxg + 




where 



f 



a 



(31) 



(32) 



The corresponding equations for user 2 are identical to fl28l) - fl3Tl) except for 
interchanging the indices 1 and 2. The free-energy is given by 



K 



/?/ = § 1- 2 + ^ 5^ (ln(l + xA)l^, - ^ E ( 1 + n 



i=l 



i=l 
2 C 



nn(i+^^*9 ) • (33) 



i=l 1=1 



For the ferromagnetic solution equation fl2Tl) results in / = 0.25. Indeed, for the MIMO 
Gaussian channel studied in this paper, we always have that the ferromagnetic free 
energy is given by / = l/2a. 

By iteratively solving the saddle point equations we obtain quantities of interest 
for this case. The free and internal energies, for L = 2 and i? = 1/4 {K = 4 and C = 3) 
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Figure 2. Free energy and internal energy in the MAC case for L — 2 represented 
by tlie solid and dashed lines, respectively; the results have been obtained by iterating 
the saddle point equations (jl5[) from arbitrary initial conditions. The dot-dashed 
line shows the theoretical limit obtained from equation ([27| and the dotted line the 
thermodynamical transition point. The entropy as a function of the noise level is shown 
in the inset; the point where the entropy becomes negative marks the emergence of 
metastable states and the dynamical transition point, while the point where it crosses 
back the zero entropy line marks the thermodynamical transition noise value. 

and at the Nishimori temperature, are represented in figure [2] by the sohd and dashed 
hues, respectively; Shannon's theoretical threshold is given by cr^ = 2, indicated by 
the dot-dashed line. The point where the free energy differs from the internal energy, 
which corresponds to the overlap changing from 1 to lower values, marks the dynamical 
transition point. The thermodynamical transition point is identified by the crossing 
of the two energies and is denoted by the dotted line. The entropy function, shown 
in the inset plotted against the noise level, also helps to identify the dynamical and 
thermodynamical transition points (where the entropy becomes negative and where it 
crosses back the coordinate axis, respectively). Both points are below Shannon's limit. 

Table [2] shows the results for L = 1,2,3 senders. The second column gives 
the theoretical limit obtained from equation ( fT4l) : it shows the deterioration in 
performance as the number of senders increases. The deterioration is also evident 
in the results obtained by numerical results obtained using the RS ansatz given by 
the dynamical transition (second column) and the thermodynamical transition (third 
column). Contrary to the single-sender case the difference between the transition points 
decreases with increasing L; this reflect the fact that additional inputs seem to increase 
the number sub-optimal solution states (and hence reduce their free energy and affect the 
thermodynamical transition point) but have a lesser effect on the onset of the metastable 
states. 
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Table 2. Comparison between the Shannon's hmit, the dynamical transition point 
and the thermodynamical transition for the MAC case (O — 1). 



L 


Shannon's 


Dynamical 


Thermodynamical 




Limit 


Transition 


Transition 


1 


2.41 


1.59 


2.24 


2 


2.00 


1.32 


1.66 


3 


1.64 


1.24 


1.45 



There are two possible scenarios one may consider in the case of a large number 
of users [L oo). The first is the random interference scenario. Due to the well- 
known isomorphism between CDMA and MIMO channels, this case is exactly the one 
calculated in [9] if one rescales Sji = Sji/\/L where the sji are i.i.d. random variables 
with zero mean, unit variance and vanishing odd moments. The second scenario is the 
deterministic interference case, where the matrix S is not random. This scenario is of 
little interest as the capacity grows with the logarithm of the number of users while the 
transmitted information grows linearly with the number of transmitters; the capacity 
per user goes to zero in this limit, rendering the communication infeasible. 



6. Interference Channel 



Interference channel [I5] refers to a scenario where several transmitters send data 
simultaneously to an equal number of receivers; the transmission from a given 
transmitter to the corresponding receiver is corrupted by (small) interference from all 
other transmitters. The receivers can then communicate with each other to optimally 
extract the original messages. Some sensor networks are among the most well known 
exemplars of systems that could be modelled by an interference channel. 

In the following, we will study two basic types of interference channels, the 
symmetric and the asymmetric case. For simplicity, we limit the number of transmitters 
and receivers considered here to L = O = 2; this will make the interpretation of 
the results easier and more transparent. Both channels are depicted in figure [31 The 
symmetric case corresponds to the transmitters sending messages to both receivers (left 
picture) while in the asymmetric case only the first transmitter sends a message to the 
first receiver while the second transmitter sends a message to both receivers. 



6.1. The Symmetric Interference Channel 

We first study the case L = O = 2 with a symmetric interference matrix 

S={1 [], (34) 
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Symmetric Interference Channel Asymmetric Interference Channel 



Figure 3. Diagram representing the symmetric (left) and asymmetric (right) 
interference channels. The first and second transmitters and receivers are denoted 
by ti, t2 and ri, r2, respectively. Arrows represent the transmitted messages and the 
double line bewtween the receivers indicates joint decoding. 



where < e < 1. The corresponding capacity can be derived using equation ( IT^ to 
obtain 

1 



\ ^ 2(1 + a ^ {1-er 



The RS saddle point equations are given by 



K-l 



(35) 



(36) 



'C-l 



7ri[xi) = { 5 { xi — tanh I atanhx'^ H ^—={ri + er2 

[ 1=1 

1 - tanh(||) tanh (^^^^ + Eti atanh^ 



+ -In 
2 



1 + tanh(fl) tanh ( ^^S^ + ^^=i atanhxa 




where 



1 + 6 

V2-^ 



1,2. 



(37) 



(3^ 



The corresponding equations for 7r2 and 7r2 are similar to those of tti and tti and can be 
obtained by interchanging the indices 1 and 2. 

Note that the same scaling as in the MAC case is necessary here due to the 
interference. However, for e = 0, this scaling should be omitted as the interference 
vanishes, leaving two separate Gaussian channels. 

The overlaps are given by 

rf,= (sgn(p))^, (39) 



c 



V{p) = (dip — tanh I atanhx^^ H ^-^{ri + er2 



1=1 



2 



1 - tanh(fl) tanh ( ^^S^ + ^^=i atanh4 



1 + tanh(f ) tanh ( ^^^^^ + Eti atanh4 



/3(eri+r2) 




(40) 
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The free-energy / is 

/5/ = 1^ 2 + 5^ (ln(l + xA))^,, - E ( 1 + n 



c . c c ^ ' ^ ^ 





r2 - 



m=l 

en + T2 



V2 



(41) 



Accordingly, the free-energy of the ferromagnetic solution fl?I]) is / = 0.5, as for 
the simple Gaussian channel. 

We solved numerically the saddle point equations (136|) and calculated quantities of 
relevance in this case. The graphs for the overlap, entropy and energy are qualitatively 
the same as in the other two cases, with a similar behaviour with the appearance of 
both dynamical and thermodynamical transition points before Shannon's limit. 

Figures m and [5] show the field distributions 7t{x) and n^x), respectively, for four 
different values of the noise level in the RS ansatz; with e = 1.0, (3 = 1 and i? = 1/4 
[K = 4, C = 3). It should be noticed that, before the dynamical transition point, 
these distributions are delta functions centred at 1, corresponding to the ferromagnetic 
solution fl?Il) . The plotted distributions are histograms with 500 bins for 40000 fields. 
In figure H] we see how the tt distribution changes slowly from the delta function in 1 to 
a delta function in zero, which is the solution for a'^ ^ oo. The if distribution depicted 
in figure [5] changes abruptly from the delta function in 1 to a highly peaked asymmetric 
distribution around zero (paramagnetic solution) when the dynamical transition point 
is crossed. Looking at the values on each of the graphs, it is visible how the scales 
increase very fast as the noise level attains higher values. 

If one keep a constant code rate i? = 1/4 but allows e to vary, one obtains the 
dependence of the threshold noise as a function of e, depicted in figure [6] (for (3 = 1, 
K = 4 and C = 3). Both dynamical (dashed line) and thermodynamical transition 
values (dashed-dotted line) are upper bounded by the theoretical limit. Although this 
may seem counterintuitive, the communication resilience against noise increases with 
the interference level. This can be understood in the case of joint detection by noting 
that the increased interference provides more information about the other transmitters, 
such that higher levels of noise can be tolerated by joint decoding. 

For large O with L ~ C(l) or large L with O ~ C(l), the results should approach 
those obtained for large number of users in the single transmitter and in the MAC case, 
respectively. The behaviour must be dictated by the value of the system load a. In 
this case, we expect the results to cross from a behaviour similar to the one of a MAC 
channel for a > 1 to one that resembles the single transmitter case for a < 1. We are 
currently working on the analytical and computational aspects of this last well 
as on the case of large O and L values while keeping the ration L/0 ^ C>{1) finite. 
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Figure 4. Profile of the tt distribution. The plots are histograms with 500 bins for 

a population of 40000 fields. The noise level value cr^ is indicated in eaeh graph. All 
noise levels are above the dynamical transition point; below the transition point the 
distribution is a delta function d{x—l). Note how the distribution changes slowly from 
S{x — 1) to S{x) as the noise level increases. 




Figure 5. Profile of the it distribution. The plots are histograms with 500 bins for 
a population of 40000 fields. The noise level value is indicated in each graph. All 
noise levels are above the dynamical transition point; below it the profile distribution 
is simply a delta function S{x — 1). In this case, the profile changes abruptly from 
5{x — 1) to an asymmetric distribution centred at a; = and diverges rapidly to 5{x) 
with the increasing noise. 



Statistical Mechanics of MIMO Channels 16 
6.0 

5.0 

4.0 

a 3.0 

2.0 

1.0 

^•ft.O 0.2 0.4 0.6 0.8 1.0 

e 

Figure 6. Transition points and theoretical limits as a function of the interference level 
e. The solid line represents the theoretical limit obtained from information theoretical 
methods; the dashed-dotted and dashed lines correspond to the thermodynamical and 
dynamical transition points, respectively. 
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6.2. The Asymmetric Interference Channel 



A variant of the interference channel discussed in section [6?T| for the case of L = O = 2, 
is the asymmetric interference channel. This realisation of the interference channel 
is highly relevant to cases where receivers are distributed at random and experience 
different noise levels, for instance, in the case of sensor networks. The interference 
matrix is asymmetric in this case and takes the form (for L = O = 2) 

1 f 
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with < e < 1. The corresponding capacity is now (again by iHM ] 
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The RS saddle point equations are given by 
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(46) 



(47) 



In this case, the scahng l/yL (in this case L = 2, although the treatment can be 
extended to include a general number of sources) appears only in the first receiver, as 
it is being affected by the interference. 

The overlaps are given by 

ci, = (sgn(p,))p,, ^ = 1,2, (48) 
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The free-energy / is obtained from 
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with the free energy of the ferromagnetic solution / = 0.5. 

The numerical solution of dH]) with e = 1.0, /5 = 1 and i? = 1/4 (/sT = 4, C = 3) 
leads to the results depicted in figure [71 The top plot shows the overlaps of the solutions 
obtained for both receivers. Interestingly, the overlaps for the two receivers behave 
significantly differently in spite of the fact that messages are decoded jointly; this 
one of the striking features of the asymmetric interference channel. The overlap for 
both receivers is one up to the point where the first receiver (thick continuous line), 
which experiences interference effects, exhibits a dynamical transition which signals 
the practical noise threshold for this system. The same point can be identified in the 
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entropy plot (bottom right) as the point where the entropy becomes negative. This 
point is very far from Shannon's limit (dotted line) ~ 7.56; this can be explained by 
the additional metastable states introduced by the asymmetric interference. Note that 
the first receiver undergoes a (dynamical) transition before the second receiver (thick 
dashed line) that does not suffer from interference; this is in spite of the fact that the 
messages are decoded jointly. 

However, the dynamical transition point for the second receiver introduces an 
unexpected behaviour at the first receiver. When the overlap for the second receiver 
drops to sub-optimal levels, the first receiver exhibits a sudden increase in its decoding 
overlap. This behaviour may be understood by examining the average overlap, which can 
be viewed as the overlap for the entire system. We see that the system's overlap suffers 
a second transition at this point, although the average overlap continues to decrease 
monotonically; the system as a whole has a certain amount of retrievable information 
which keeps decreasing with the noise level. 

The above result shows that the way information is distributed among the receivers 
can be highly non-trivial. It also shows that for systems with many users, the 
thermodynamical transition point is determined mostly by the weakest node (which 
experiences the highest levels of interference) and may lead practical fimits that are 
very far from the Shannon bound. 

7. Conclusions 

We investigated the properties of coded Gaussian MIMO channels using methods 
of statistical mechanics. The problems investigated relate to the cases of a single 
transmitter, multiple access and interference in the case of multiple receivers and 
transmitters. In all cases, transmissions are coded using LDPC error-correcting codes. 

The method used in the analysis, the replica approach, enables one to obtain typical 
case results that complements the theoretical bounds reported in the information theory 
literature. The numerical results obtained for particular MIMO channels and parameter 
values are presented and contrasted with the information theoretical results. 

MIMO channels are characterised by an interference matrix S which mixes inputs 
from the various transmitters to provide the messages at the receiving end. We examine 
cases where the interference matrix is deterministic. This requires the introduction of a 
non-trivial scaling in order to obtain meaningful results. 

The results obtained provide characteristic, typical case, results in all cases. For the 
single transmitter and MAC problems we show both dynamical and thermodynamical 
transition points as functions of the number of receivers and transmitters, respectively. 
We see that the gaps between the practical and theoretical thresholds (dynamical and 
thermodynamical transitions, respectively), and the gap between them and Shannon's 
limit, increase with the number of receivers in the single transmitter case and decrease 
with the number of transmitters in the MAC case. 

In the single transmitter case, this results from the increase in the number of 
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Figure 7. Numerical integration of saddle point equations for the asymmetric 
interference channel. The upper plot shows the overlap for the first receiver (thick 
continuous line), which suffer the effects of interference, the second receiver (thick 
dashed line) and the average overlap for the entire system (thin continuous line). The 
Shannon limit for the system is depicted by the dotted vertical line. At the bottom, 
the left graph shows the free-energy (continuous line) and the internal energy (dashed 
line) for the entire systems while the right graph shows the entropy values obtained 
under the RS ansatz. 



variables and consequentially the increase in the number of metastable states. The point 
where metastable solutions emerge determines the dynamical transition point (practical 
threshold), while the number of metastable states affects the thermodynamic transition 
point. The increasing number of transmitters in the MAC case enables one to effectively 
reduce the noise level by averaging over a higher number of random and independent 
noise sources. 

The comparison with theoretical limits for the single transmitter case reveals an 
important feature of multiuser channels as to how the available information is used. 
The huge gap between the transition points and Shannon's limit is indicative of a poor 
use of resource, and suggests network coding as a measure to achieve a good use of 
resource; without it, the system's efficiency remains below the achievable theoretical 
limit for sending the same message repeatedly via a simple Gaussian channel. One 
possible solution that we are currently investigating is the use of fountain codes [231 121] 
for making a more efficient use of the available resource. 

The main result in the symmetric interference channel case is the increase in both 



Statistical Mechanics of MIMO Channels 



20 



dynamical and thermodynamical transition points as a function of the interference 
parameter e. Results for low e values are similar to the case of separate Gaussian 
channels; as e increases, both values come closer to Shannon's theoretical limit with the 
thermodynamical transition point showing a stronger increase. This could be explained 
by the increase of (mixed) information in comparison to the noise level; this information 
can be decoded jointly, with an effectively lower noise level. The more moderate 
increase in the practical threshold (dynamical transition) is due to the difficulty in 
jointly decoding the various sources in practice due to the emergence of metastable 
states. 

In the asymmetric case we found a striking different behaviour of the system. The 
new feature observed is the second transition suffered by the system as a whole. We 
also detected a surprising behaviour of the receiver which experiences interference; in 
spite of the joint decoding, the information available to it is suppressed by the second 
receiver. Only when the second receiver stops decoding perfectly, the performance of 
the first receiver improves. 

An interesting extension, of significant practical relevance, would be the extend 
the LDPC coding framework to complex MIMO channels, where circular noise is 
considered [25]. Another as well as the possible extension is the case of a large number 
of senders and receivers where the ration between them remains finite. The study of 
these and other related problems is underway. 
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Appendix A. Replica Symmetric Calculations 



^From the partition function ( 1121) . we can write the averaged replicated partition 
function Z„ = (^")Ai,...,A£,r,t 
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where the multiplicative constants come from the normalisation of the probability 
distributions in the outside average and we defined Tjo = ti- Following [H], we have 
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and the variables mj assume all integer values for the index i from to + 1 
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and using integral representations for the delta functions, we can write 
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and integrating over the variables Z^, the /i indices factorise and we obtain 



Z„ = / DqDq exp 



MLf{q,q) 



with 



/(g,g) = -ln7 + — ^ 



El- 

1=1 



1 ^ 1 

- yEE^'^i^'^^ + yln$, 



i=l u>i 



and 



$ = A 



X exp 



E ^ r^- - E ^i^'^A 

j=l 3 \ i=l / 



Statistical Mechanics of MIMO Channels 
Using the replica symmetric (RS) ansatz 

where 
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Inserting the result in $ and summing over the zero-th replicas we have 
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Let us consider L = 1. Then 
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Derivations with respect to go and go give 2gogo = C and functional derivatives with 
respect to 7r(x) and 7r(x) give equations (ITS]) of section HI 

Appendix A. 2. MAC 
In this case, = 1, 
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and the corresponding extremisation, including the necessary normalisation, gives 
equations (!28l) of section [51 

Appendix A. 3. Interference Channel 

The MIMO case with L = O = 2 can be viewed as an interference Gaussian channel 
where the receivers cooperate with each other to decode the received message. In this 
case 
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Extremisation with respect to vTj, i = 1, 2 results in 
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where we defined 
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Equating to zero we obtain 
7ri(a;i) = {5{xi - I 
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where 
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The final equations with the interference normahsation are aheady given in section 
E] for both cases of a symmetric (subsection I6.ip and an asymmetric (subsection 16.21) 
interference channels. These equations can be easily generalised for any number of L 
and O values. In numerical calculations, however, the numerical errors occurring due to 
the introduction of additional fields in this direct form are difficult to control. Clever 
algebraic manipulations are necessary to keep these errors under control in order to 
obtain accurate results. 



